Robotic surgery system including position sensors using fiber bragg gratings

ABSTRACT

A surgical instrument is provided, including: at least one articulatable arm having a distal end, a proximal end, and at least one joint region disposed between the distal and proximal ends; an optical fiber bend sensor provided in the at least one joint region of the at least one articulatable arm; a detection system coupled to the optical fiber bend sensor, said detection system comprising a light source and a light detector for detecting light reflected by or transmitted through the optical fiber bend sensor to determine a position of at least one joint region of the at least one articulatable arm based on the detected light reflected by or transmitted through the optical fiber bend sensor; and a control system comprising a servo controller for effectuating movement of the arm.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. Patent Application No.11/491,384 (filed Jul. 20, 2006) now U.S. Pat. No. 7930065 , whichclaims the benefit of provisional U.S. patent application Ser. No.60/755,157 (filed Dec. 30, 2005), both of which are incorporated hereinby reference.

BACKGROUND

In robotically-assisted or telerobotic surgery, the surgeon typicallyoperates a control device to control the motion of surgical instrumentsat the surgical site from a location that may be remote from the patient(e.g., across the operating room, in a different room or a completelydifferent building from the patient) or immediately adjacent to thepatient. The controller usually includes one or more manually-operatedinput devices, such as joysticks, exoskeletal gloves or the like, whichare coupled (directly or indirectly) to the surgical instruments withservo motors for articulating the instruments at the surgical site. Theservo motors are typically part of an electromechanical device orsurgical manipulator that supports and controls the surgical instrumentsthat have been introduced directly into an open surgical site or throughtrocar sleeves into a body cavity, such as the patient's abdomen. Duringthe operation, the surgical manipulator provides mechanical articulationand control of a variety of surgical instruments, such as tissuegraspers, needle drivers, electrosurgical cautery probes, etc., thateach perform various functions for the surgeon, e.g., holding or drivinga needle, grasping a blood vessel, or dissecting, cauterizing orcoagulating tissue. A surgeon may employ a large number of differentsurgical instruments/tools during a procedure.

Robotic manipulators, particularly small ones of the type used inminimally-invasive surgery, generally need to transmit actuation forcesand sensor information some distance through the structure of therobotic device, because the mass and size of presently-availableactuators (e.g., motors) and sensors (e.g., position encoders) are toolarge to permit them to be directly located where the force and sensingis needed. In a mechanical device of a type used in minimally-invasivesurgery, it is desirable to provide (a) significant force and (b)positioning precision at the tip of a small structure. However, themotors and sensors used are typically too large and heavy to be placedinside the patient's body. In addition, safety concerns maketransmission of electrical power to actuators or sensors located insidethe patient's body problematic. Therefore, in these roboticapplications, it is desirable to transmit the actuation forces formultiple degrees of freedom over significant distances, and the desireto minimize the size of incisions places a premium on thecross-sectional area used for the structure and the transmission of theactuation forces. In addition, it is desirable to keep the transmissionmechanism for these forces as stiff and friction-free as possible, sothat the proximally sensed position of the proximal actuator may be usedas a fair representation of the distally positioned joint.

There are many means known to transmit force over a distance. Someexamples include: cables in tension; rods or tubes in compression;torsion; and hydraulic or pneumatic pressure. None of these forcetransmission means are ideal for minimally-invasive surgery, andtherefore there are practical limits to the precision with which adistally mounted structure member can be moved, and the force it canprovide. For example, cables may stretch, thereby leading to errors inthe distal structure position relative to the position of the proximaldriving mechanism. Compliant cables may also have significant frictionwith the supporting structure. Friction is notoriously difficult tomodel, may behave in a time-varying way and is dependent upon any numberof variables, including the position (and past history of the position)of any intermediate joints in the structure, and when combined withcompliant actuation can lead to stick-slip behavior. These errors limitthe utility of the robot, limit the number of degrees of freedom thatcan be controlled remotely through a small structure, limit the use ofthe robot's state to provide haptic feedback, and are particularlysevere for a flexible surgical robot which must accurately control manydegrees of freedom of actuation at the end of a long, flexiblestructure.

Feedback regarding the actual position of a structure is a commonlyapplied mechanism for compensating for these errors. If the position ofthe distal part is accurately measured with minimal delays, controlalgorithms can compensate for many of the deficiencies of the forcetransmission mechanisms. While there are limits to the compensation thatcan be done with feedback, those limits cannot even be approachedwithout accurate, timely information about the positions of thedistally-mounted parts. The main restriction on the applicability offeedback in these types of robotic arms is the lack of an effectivemeans of determining the position of parts mounted at the distal end ofthe structure.

Two methods of determining the position of a structure areproprioception and exteroception. Proprioception refers to the internalsensing of the position of the structure and exteroception refers to theexternal sensing of the position.

A proprioceptive system may monitor the motors actuating the movement ofeach joint in a robotic structure. By monitoring the degree of movementof each motor, the expected movement of each corresponding joint can beestimated. For example, the da Vinci Surgical System by IntuitiveSurgical, Inc., of Sunnyvale, Calif. utilizes motor sensors (e.g.,encoders) for position feedback. The encoders are co-located with theactuators, which are positioned outside of the patient body and drivethe structure's joints through a cable mechanism. However, this methodallows the position control loop to compensate only for error sourcesthat occur between the actuator and the encoder. Error sources distal tothe encoder are not observed by the encoder and thus cannot becompensated. This arrangement does not provide for compensation oferrors introduced by any structures or drivetrain mechanisms that aredistal to the encoders. In order to estimate the position of distaljoints, the cable mechanism is modeled as infinitely stiff, so the motorposition is assumed to be a reliable indication of the actual jointposition. If the drivetrain between the actuator and the joint is notsufficiently stiff or other errors exist between the joint and theencoder, there will be a difference between actual joint orientation andthe expected orientation based on the motor position. These joints areoften mounted serially, so that the relative orientation of each of thelinks in the structure must be known and transformed to determine theposition of the distal end of the structure. As a result, errors in thesensing of the orientations of intermediate links may compound along theway.

In other cases, position sensors may be positioned directly at thejoints. This arrangement may be effective for larger structures, such asthe Canadarm2 used on the International Space Station, and industrialrobots. In robotically-assisted surgical systems having many degrees offreedom near the distal end, joint encoders do not offer much benefit asthey are too large to be placed where the most problematic joints arelocated, and the connections required to transmit their position databack to the controller compete for space with the force-transmittingapparatus inside the arm.

Tachometers and accelerometers have been used, typically to estimate therobot's configuration at successive time steps and for force-controlapplications. These systems also do not provide complete position dataand therefore cannot be used to compensate for some types of errorsdescribed above.

In systems which utilize exteroception, externally observed positionfeedback may be used to determine the position of a robotic structure.

For example, GPS-based location (especially differential GPS) has beenused in outdoor environments, but typically can only be applied to largestructures due to the relatively poor spatial resolution.

Field-based sensing can be accomplished using either AC or DC magneticfields. In this approach, a small sensing element is placed at the tipof the structure to be monitored, and a magnetic field generatedexternally. While these systems can achieve good accuracy under idealconditions, the accuracy rapidly degrades when metallic objects arenearby (as is the case in most surgical applications), and an externalapparatus is needed to generate the field. In addition, these sensors donot directly encode the positions of the joints or links in a structureunless a sensor is provided on each link. Therefore, differentconfigurations of the structure which result in the same position of thesensor are not distinguishable, which limits the use of this type ofdata for feedback compensation as described above.

Another approach is to utilize a dedicated set of unloaded cables,attached to the movable member at the distal end and to an encoder orposition measurement device at the proximal end. Although unloaded,these cables are still subject to the same bending at the intermediatejoints as are the actuation devices, and to provide feedback for manydegrees of freedom, many cables must be used, requiring a largerstructure.

Methods have been used involving a combination of sensing of theexternal environment, locating landmarks, and using this information tosimultaneously construct a map of the environment and localize therobot. However, these systems are only applicable to larger andslower-moving systems, such as mobile robotic platforms. In the surgicalapplications considered here, these methods may be undesirable becausethe landmarks (e.g. patient anatomy) are not well defined and may changeposition as a result of disease states or during the surgicalmanipulations.

Techniques known as visual-serving have been proposed, in which a cameramounted on the end of the robot arm is used in combination with jointencoders to control the position of the robot end-effector with respectto a tracked object in the field of view of an imaging device. Thisapproach suffers from the need to provide joint position feedback andidentifiable landmarks in the viewed scene, which are also problematicin surgical applications.

These techniques suffer from various deficiencies when utilized inrobotic surgical applications and do not provide the type of informationdesired for implementing closed-loop control of such robotic mechanismsand compensating for the deficiencies in the force-transmission meansdescribed above. In addition, these techniques may be complex andexpensive to implement.

Accordingly, it would be desirable to provide systems and methods fordetermining the position of a surgical instrument at a surgical site ona patient. In particular, it would be desirable for these systems andmethods to provide real-time position feedback to the control system fora robotic surgical instrument.

SUMMARY

In accordance with embodiments of the present invention, a surgicalinstrument is provided, comprising: at least one articulatable armhaving a distal end, a proximal end, and at least one joint regiondisposed between the distal and proximal ends; an optical fiber bendsensor provided in the at least one joint region of the at least onearticulatable arm; a detection system coupled to the optical fiber bendsensor, said detection system comprising a light source and a lightdetector for detecting light reflected by or transmitted through theoptical fiber bend sensor to determine a position of at least one jointregion of the at least one articulatable arm based on the detected lightreflected by or transmitted through the optical fiber bend sensor; and acontrol system comprising a servo controller for effectuating movementof the arm.

In accordance with other embodiments of the present invention, asurgical instrument is provided, comprising: at least one elongate armcomprising a passively-bendable region and an actively-controlledbendable region including at least one joint region; a control systemcomprising a servo controller for effectuating movement of the at leastone joint region; an optical fiber bend sensor provided in at least oneof the passively-bendable region and the actively-controlled bendableregion; and a detection system coupled to the optical fiber bend sensor,said detection system comprising a light source and a light detector fordetecting light reflected by or transmitted through the optical fiberbend sensor.

Other features and aspects of the invention will become apparent fromthe following detailed description, taken in conjunction with theaccompanying drawings which illustrate, by way of example, the featuresin accordance with embodiments of the invention. The summary is notintended to limit the scope of the invention, which is defined solely bythe claims attached hereto.

DESCRIPTION OF THE DRAWINGS

FIG. 1 shows a robotic surgical system, in accordance with embodimentsof the present invention.

FIG. 2 shows a simplified block diagram of a surgical instrument, inaccordance with embodiments of the present invention.

FIG. 3 is a simplified cross-sectional view of an exemplary opticalfiber, which may be used in accordance with embodiments of the presentinvention.

FIG. 4 is an exemplary block diagram of a first pair of opposing coresand the associated interrogation mechanisms.

FIG. 5 is a simplified block diagram of another embodiment of thepresent invention.

FIG. 6 is a perspective view of a surgical instrument, in accordancewith embodiments of the present invention.

FIG. 7 is a simplified perspective view of a surgical instrument, inaccordance with embodiments of the present invention.

FIG. 8 is a simplified block diagram illustrating a multi-input feedbackloop, in accordance with embodiments of the present invention.

FIGS. 9A-9B are simplified block diagrams illustrating top and sideviews of a portion of a surgical instrument, in accordance withembodiments of the present invention.

FIG. 10 illustrates a possible spectrum that may result when only partof a sensor region is contained with a joint region.

DETAILED DESCRIPTION

In the following description, reference is made to the accompanyingdrawings which illustrate several embodiments of the present invention.It is understood that other embodiments may be utilized and mechanical,compositional, structural, electrical, and operational changes may bemade without departing from the spirit and scope of the presentdisclosure. The following detailed description is not to be taken in alimiting sense, and the scope of the embodiments of the presentinvention is defined only by the claims of the issued patent.

The terminology used herein is for the purpose of describing particularembodiments only and is not intended to be limiting of the invention.Spatially relative terms, such as “beneath”, “below”, “lower”, “above”,“upper” and the like, may be used herein for ease of description todescribe one element or feature's relationship to another element(s) orfeature(s) as illustrated in the figures. It will be understood that thespatially relative terms are intended to encompass differentorientations of the device in use or operation in addition to theorientation depicted in the figures. For example, if the device in thefigures is turned over, elements described as “below” or “beneath” otherelements or features would then be oriented “above” the other elementsor features. Thus, the exemplary term “below” can encompass both anorientation of above and below. The device may be otherwise oriented(rotated 90 degrees or at other orientations) and the spatially relativedescriptors used herein interpreted accordingly.

As used herein, the singular forms “a”, “an” and “the” are intended toinclude the plural forms as well, unless the context indicatesotherwise. It will be further understood that the terms “comprises”and/or “comprising” specify the presence of stated features, steps,operations, elements, and/or components, but do not preclude thepresence or addition of one or more other features, steps, operations,elements, components, and/or groups thereof.

Described herein are embodiments of a multi-component system, apparatus,and method for performing robotically-assisted surgical procedures on apatient, particularly including open surgical procedures, neurosurgicalprocedures, such as stereotaxy, and endoscopic procedures, such aslaparoscopy, arthroscopy, thoracoscopy and the like. The system andmethod of the present invention may be particularly useful as part of arobotic surgical system that allows the surgeon to manipulate thesurgical instruments through a servo controller from a location remotefrom the patient. To that end, the manipulator apparatus of the presentinvention may be driven by a controller to form a telepresence system.The systems may comprise a master-slave configuration in which thecontroller and manipulator apparatus are separate devices, or maycomprise an augmented system in which the controller and manipulator areprovided as part of a single device. A description of a suitableslave-master system can be found in U.S. patent application Ser. No.08/517,053, filed Aug. 21, 1995, the complete disclosure of which isincorporated herein by reference for all purposes.

Referring to FIG. 1, a robotic surgical system 2 is illustratedaccording to an embodiment of the present invention. A similar system isdescribed in U.S. provisional patent application Ser. No. 60/755,157,filed on Dec. 30, 2005, entitled “Modular Force Sensor,” the disclosureof which is incorporated herein in its entirety. As shown in FIG. 1, therobotic system 2 generally includes one or more surgical manipulatorassemblies 4 mounted to or near an operating table O, and a controlassembly 6 for allowing the surgeon S to view the surgical site and tocontrol the manipulator assemblies 4. The system 2 will also include oneor more viewing scope assemblies 19 and a plurality of surgicalinstruments 20 adapted to be removably coupled to manipulator assemblies4 (discussed in detail below). In some cases, the scope assemblies 19may be integral with the surgical instruments 20. The robotic system 2may include one or more manipulator assemblies 4 and preferably three orfour manipulator assemblies 4. The exact number of manipulatorassemblies 4 will depend on the surgical procedure and the spaceconstraints within the operating room among other factors. As discussedin detail below, one of the manipulator assemblies 4 may operate aviewing scope assembly 19 (e.g., in endoscopic procedures) for viewingthe surgical site, while the other manipulator assemblies 4 operatesurgical instruments 20 for performing various procedures on the patientP.

Control assembly 6 may be located at a surgeon's console C which isusually located in the same room as operating table O so that thesurgeon may speak to his/her assistant(s) A and directly monitor theoperating procedure. However, it should be understood that the surgeon Scan be located in a different room or a completely different buildingfrom the patient P. Control assembly 6 generally includes a support 8, amonitor 10 for displaying an image of the surgical site to the surgeonS, and one or more control device(s) 12 for controlling the manipulatorassemblies 4. The control device(s) 12 may include a variety of inputdevices, such as joysticks, gloves, trigger-guns, hand-operatedcontrollers, voice recognition devices or the like. Preferably, thecontrol device(s) 12 will be provided with the same degrees of freedomas the associated surgical instruments 20 to provide the surgeon withtelepresence, or the perception that the control device(s) 12 areintegral with the instruments 20 so that the surgeon has a strong senseof directly controlling instruments 20. In some embodiments, the controldevices 12 are manual input devices which move with six degrees offreedom, and which may also include an actuatable handle for actuatinginstruments (for example, for closing grasping saws, applying anelectrical potential to an electrode, or the like).

Position, applied force, and tactile feedback sensors (not shown) mayalso be employed on instrument assemblies 20 to transmit informationregarding position, applied force, and tactile sensations from thesurgical instrument back to the surgeon's hands as he/she operates therobotic system. One suitable system and method for providingtelepresence to the operator is described in U.S. patent applicationSer. No. 08/517,053, filed Aug. 21, 1995, which is incorporated byreference herein in its entirety.

Monitor 10 may be operatively coupled to the viewing scope assembly 19such that an image of the surgical site is provided adjacent thesurgeon's hands on surgeon console C. Preferably, monitor 10 willdisplay on a display 18 an image of the surgical site and surgicalinstruments. The display 18 and the master control devices 12 may beoriented such that the relative positions of the imaging device in thescope assembly and the surgical instruments are similar to the relativepositions of the surgeon's eyes and hands so the operator can manipulatethe end effector and the hand control as if viewing the workspace insubstantially true presence. By true presence, it is meant that thepresentation of an image is a true perspective image simulating theviewpoint of an operator that is physically manipulating the surgicalinstruments 20.

As shown in FIG. 1, a servo controller 16 is provided for controllingthe mechanical motion of the manipulator assemblies 4 in response tomovement of the master control devices 12. In some embodiments, servocontroller 16 may provide force and torque feedback from the surgicalinstruments 20 to the hand-operated control devices 12. To operateeffectively with this system, manipulator assemblies 4 may be providedwith a relatively low inertia, and the drive motors may have relativelylow ratio gear or pulley couplings. Any suitable conventional orspecialized servo controller may be used.

Servo controller 16 may be separate from, or integral with manipulatorassemblies 4. In some embodiments, the servo controller 16 andmanipulator assemblies 4 are provided as part of a robotic arm cartpositioned adjacent to the patient's body. The servo controller 16transmits signals instructing the manipulator assemblies 4 to moveinstruments having shafts which extend into an internal surgical sitewithin the patient body via openings in the body. Robotic surgerysystems and methods are further described in U.S. Pat. No. 5,797,900,filed on May 16, 1997, issued on Aug. 25, 1998, U.S. Pat. No. 6,132,368,filed on Nov. 21, 1997, issued on Oct. 17, 2000, U.S. Pat. No.6,331,181, filed on Oct. 15, 1999, issued on Dec. 18, 2001, U.S. Pat.No. 6,441,577, filed on Apr. 3, 2001, issued on Aug. 27, 2002, U.S. Pat.No. 6,902,560, filed on Jan. 6, 2004, issued on Jun. 7, 2005, U.S. Pat.No. 6,936,042, filed on Apr. 16, 2002, issued on Aug. 30, 2005, and U.S.Pat. No. 6,994,703, filed on Dec. 4, 2002, issued on Feb. 7, 2006, thefull disclosures of which are incorporated herein by reference. Asuitable robotic surgical system currently in use is the da Vinci SSurgical System by Intuitive Surgical, Inc.

Each of the manipulator assemblies 4 may support a surgical instrument20 and may comprise a series of manually articulatable linkages,generally referred to as set-up joints, and a robotic manipulator. Themanipulator assemblies 4 enable the instrument 20 to be rotated around apoint in space, as more fully described in issued U.S. Pat. Nos.6,331,181, and 5,817,084, the full disclosures of which are incorporatedherein by reference. The robotic manipulators may pivot the instrumentabout a pitch axis, a yaw axis, and an insertion axis (which is alignedalong a shaft of the instrument). The instrument has still furtherdriven degrees of freedom as supported by the manipulator, includingsliding motion of the instrument along the insertion axis.

The robotic manipulator assemblies 4 may be driven by a series ofactuators (e.g., motors). These motors actively move the roboticmanipulators in response to commands from the servo control 16. Themotors are further coupled to the surgical instrument so as to rotatethe surgical instrument about the insertion axis, and to articulate awrist at the distal end of the instrument about at least one, and oftentwo or more, degrees of freedom. Additionally, the motors can be used toactuate an articulatable end effector of the instrument for graspingtissues in the jaws of a forceps or the like. The motors may be coupledto at least some of the joints of the surgical instrument using cables,as more fully described in U.S. Pat. Nos. 6,331,181, and 5,792,135, thefull disclosures of which are also incorporated herein by reference. Asdescribed in those references, the manipulators will often includeflexible members for transferring motion from the drive components tothe surgical instrument. Each actuator may effectuate movement of one ormore joint members in the instrument. For endoscopic procedures, themanipulators may include a cannula, which supports the surgicalinstrument, allowing the surgical instrument to rotate and move axiallythrough the central bore of the cannula.

In accordance with embodiments of the present invention, a surgicalinstrument is provided. This surgical instrument includes an elongatebody having a positionable distal end and at least one bendable region,such as a joint region or a flexible region. An optical fiber bendsensor comprising one or more optical fibers is provided in the bendableregion of the body. Each of these optical fibers includes a Fiber BraggGrating, preferably a collinear array of Fiber Bragg Gratings. A strainsensor system comprising a light source and a light detector is used tomeasure strain in the optical fibers in order to determine a positionand shape of the body. This shape and position information can be usedto assist in controlling movement of the robotic manipulator and/orsurgical instrument. This position information may include bothtranslational and rotational position.

FIG. 2 shows a simplified block diagram of a surgical instrument 200, inaccordance with embodiments of the present invention. The instrument 200includes an elongate body comprising a plurality of body segments 212coupled to adjacent body segments 212 via joint regions 214. Each jointregion 214 may provide, one, two, or more degrees of freedom for theinstrument 200. A channel 216 passes through the elongate body 210, andan optical fiber 220 is provided within the channel 216. A sensorcontrol system 250 is coupled to a proximal end of the optical fiber220. In this embodiment, the body segments 212 are cylindrical in shapehaving a diameter of approximately 5 mm, and the fiber 220 has adiameter of approximately 200 um. In other embodiments, the shapes anddimensions of the components may vary.

Fiber optic bend sensors for determining a shape of a structure havebeen used. For example, optical fibers including Fiber Bragg Gratings(FBG) have been used in a variety of applications for providing strainmeasurements in structures in one or more dimensions. Various systemsand methods for monitoring the shape and relative position of a opticalfiber in three dimensions are described in U.S. patent applicationpublication no. 2006/0013523, filed on Jul. 13, 2005, U.S. provisionalpatent application Ser. No. 60/588,336, filed on Jul. 16, 2004, and U.S.Pat. No. 6,389,187, filed on Jun. 17, 1998, the disclosures of which areincorporated herein in their entireties.

In accordance with embodiments of the present invention, an opticalfiber, such as the optical fibers described in U.S. patent applicationpublication no. 2006/0013523, is utilized as optical fiber 220 formonitoring the shape and relative position of each body segment 212 inthe instrument 200. This information, in turn, in can be used todetermine other related variables, such as velocity and acceleration ofthe parts of a surgical instrument. By obtaining accurate measurementsof one or more of these variables in real time, the controller canimprove the accuracy of the robotic surgical system and compensate forerrors introduced in driving the component parts. The sensing may belimited only to the degrees of freedom that are actuated by the roboticsystem, or may be applied to both passive (e.g., unactuated bending ofthe rigid members between joints) and active (e.g., actuated movement ofthe instrument) degrees of freedom.

In this embodiment, the optical fiber 220 comprises three corescontained within a single cladding. Each core may be single-mode withsufficient distance and cladding separating the cores such that thelight in each core does not interact significantly with the lightcarried in other cores. In other embodiments, the number of cores mayvary or each core may be contained in a separate optical fiber.

An array of Fiber Bragg Gratings is provided within each core. EachFiber Bragg Grating comprises a series of modulations of the core'srefractive index so as to generate a spatial periodicity in therefraction index. The spacing may be chosen so that the partialreflections from each index change add coherently for a narrow band ofwavelengths, and therefore reflect only this narrow band of wavelengthswhile passing through a much broader band. During fabrication of theFiber Bragg Gratings, the modulations are spaced by a known distance,thereby causing reflection of a known band of wavelengths. However, whena strain is induced on the fiber core, the spacing of the modulationswill change, depending on the amount of strain in the core.

To measure strain, light is sent down the fiber, and the reflectedwavelength is a function of the strain on the fiber and its temperature.This FBG technology is commercially available from a variety of sources,such as Smart Fibres Ltd. of Bracknell, England. When applied to amulticore fiber, bending of the optical fiber induces strain on thecores that can be measured by monitoring the wavelength shifts in eachcore. By having two or more cores disposed off-axis in the fiber,bending of the fiber induces different strains on each of the cores.These strains are a function of the local degree of bending of thefiber. Regions of the cores containing FBGs, if located at points wherethe fiber is bent, can thereby be used to determine the amount ofbending at those points. These data, combined with the known spacings ofthe FBG regions, can be used to reconstruct the shape of the fiber. Sucha system has been described by Luna Innovations. Inc. of Blacksburg, Va.

A control system 250 is also provided for detecting the position of thesurgical instrument 200 and for utilizing that information to assist insurgical procedures. In this embodiment, the control system 250comprises a detection system 260 coupled to an imaging system 270 andthe servo controller 16. The detection system 260 is utilized forgenerating and detecting the light used for determining the position ofthe instrument 200. The imaging system 270 is used to provide thesurgeon or other operator with real-time position information for use inthe control of the instrument 200. The servo controller 16 may utilizethe position information as feedback for positioning the instrument 200.

FIG. 3 is a simplified cross-sectional view of an exemplary opticalfiber 320, which may be used in accordance with embodiments of thepresent invention. In other embodiments, other types of fibers may beused, as would be understood by one of ordinary skill in the art. Inthis embodiment, the fiber 320 includes four optical cores 330 a-330 ddisposed at equal distances from the axis of the fiber 320 such that incross-section, opposing pairs of cores 330 a-330 d form orthogonal axes,first axis 331 and second axis 332. In each core 330 a-330 d, an arrayof collinear Fiber Bragg Gratings are disposed at known positions alongthe lengths of each core 330 a-330 d such that the Fiber Bragg Gratings340 a-340 d for all four cores 330 a-330 d are aligned at a plurality ofsensor regions 350 a-350 b. FIG. 4 is an exemplary block diagram of afirst pair of opposing cores 330 a, 330 c and the associatedinterrogation mechanisms.

A bending of the fiber 220 in one of the sensor regions 350 willlengthen at least one core 330 a-330 d with respect to the opposing core330 a-330 d. Interrogation of this length differential enables the angleand radius of bending to be extracted. This interrogation may beperformed using the detection system 260, as described below.

There are a variety of ways of multiplexing the Fiber Bragg Gratings sothat a single fiber core can carry many sensors and the readings of eachsensor can be distinguished. In some embodiments, Optical FrequencyDomain Reflectometry (OFDR) may be used in which the Fiber BraggGratings, all with the same grating period, are placed along each of thecores, and each core is terminated at the proximal end with a partiallyreflecting mirror. The Fiber Bra Gratings are placed in such a way thatthe distance from each grating to the reflector is unique, which causesthe reflection spectrum of each Fiber Bragg Grating to be modulated witha unique modulation frequency, thereby allowing the individualreflection spectra to be determined. In addition, OFDR may be used tointerrogate the array of Fiber Bragg Gratings with sufficiently lowdelays such that that the bending data can be used as a feedback signalin a real-time motion control loop.

In the embodiment shown in FIG. 4, one OFDR FBG interrogation system isapplied to each of the fiber cores. This interrogation system comprisesan optical source 410 and an optical detector 420 operably coupled toeach core 330 via a coupling device 430. A broadband reflector 431 ispositioned at a location between the coupling device and the firstsensor region 350 a. The optical detectors 420 may comprise, e.g., PINphotodiodes for detecting signal reflected by the Fiber Bragg Gratings.The optical detectors 420, in turn, are coupled to OFDR demodulators 422to determine the strain in sensor region 350 of each core 330 associatedwith the detected signal. The optical source 410 used in the OFDRinterrogation system may be, e.g., a tunable laser, and provideshigh-coherence light that can be swept over a broad band of wavelengths.This light is coupled into each of the fiber cores, and the lightreflected from the core is measured as the light source is swept inwavelength. The reflected light is a sum of the reflections from each ofthe Fiber Bragg Gratings along the core, but since each reflection ismodulated with a distinct frequency (determined by the grating'sdistance from the broadband reflector), the reflection spectrum of eachgrating can be separated from the others using the data acquired in asingle scan of the core. The shift in each grating is proportional tothe strain in that core at the location of the grating, and thestrain-induced shifts of the reflection spectrum of a pair of co-locatedgratings can be subtracted as shown in FIG. 4 to give a resultproportional to the degree of bending applied to the multi-core fiber atthe axial location of the grating pair.

For embodiments which detect position based on two degrees of freedom ateach joint of the surgical instrument, at least three fiber cores 330are utilized, with an interrogation system for each fiber core 330. Inthe embodiment illustrated in FIG. 3, four cores 330 are utilized, butin FIG. 4, only a first pair of opposing cores 330 a, 330 c are shown.It will be understood that a similar interrogation system arrangementwill be provided for the second pair of opposing cores 330 b, 330 d. Fora system in which three cores are used, there are three raw data setsgenerated by the three detectors, and three OFDR calculations, each OFDRcalculation providing an output for each FBG region along the core.

As a result, the differences in strain between the Fiber Bragg Gratingsin the first pair of opposing cores 330 a, 330 c and the second pair ofopposing cores 330 b, 330 d in each sensor region 350 can be used todetermine the fiber's bending at each instrumented axial locationcorresponding to the sensor region 350. With four cores 330 a-330 ddisposed equally around the fiber's neutral axis, the strains measuredfor each pair of opposing cores 330 are subtracted. The differencebetween strains of the opposing cores 330 a and 330 c is proportional tothe degree of bending along the first axis 331, and the differencebetween strains of the opposing cores 330 b and 330 d is proportional tothe degree of bending along the second axis 332.

In the embodiment shown in FIG. 2, the instrument 200 comprises rigidbody segments 212 coupled together by discrete joint regions 214. Inthis case, because the bending of the instrument 200 will occur at thejoint regions 214, it is most desirable to position each sensor region350 so that the curvature of the fiber 220 at the joint region 214maintains a fixed axial relation to the joint angle over the entirerange of joint motion. This may be accomplished by forming the channel216 along the neutral axis of the instrument 200, so that there is noaxial displacement of the fiber 220 as the joints are moved. As aresult, it will be easier to correlate the detected strains with bendingin particular joint regions 214.

In other embodiments, it may not be desirable or practical to positionthe fiber 220 on the neutral axis of the instrument 200. If the fiber220 is positioned off-axis, then bending of the instrument 200 willchange the length of the path of the fiber 220 through the channel 216in the instrument 200. If the extent of bending is very small, then thefiber 220 may stretch or compress to accommodate the change in pathlength. Thus, the fiber 220 may have multiple fixed connection pointswith the instrument 200. This arrangement may be undesirable in somecases because increased axial loading of the fiber increases thecommon-mode signal in the fiber cores, thereby reducing the accuracy ofthe bending measurement. Considering a single pair of cores on oppositesides of the fiber's neutral axis, if there is no overall strain on thefiber, then bending at the sensor causes one FBG wavelength to shiftdown and the other to shift up by the same amount, thereby resulting ina zero common-mode signal. If, however, there is an overall strain onthe fiber, both FBG wavelengths will shift in the same direction, in theabsence of any bending at the sensor. Because each core is only a smalldistance (e.g., ˜75 μm) from the fiber's neutral axis, the strains onthe individual cores caused by bending of the fiber are quite small.Therefore, even a small axial strain on the fiber introduces a largecommon-mode signal, and gain errors between the two sensor paths mayresult in a false differential reading caused by the large common-modesignal, falsely indicating bending when the fiber is, in fact, straight.

If the extent of bending is large, then the change in path length can begreater than can be accommodated by the elasticity of the fiber 220.Thus, it may be desirable to have the fiber 220 translate axiallyrelative to the instrument 200, causing the sensor regions 350 to alsotranslate axially relative to the joint regions 214. One way to allowfor this translation is to ensure that the length of each sensor region(e.g. 350 a) exceeds the length of its associated joint region (e.g. 214a) by a sufficient amount, and that the joint regions are spacedsufficiently apart by rigid body segments 212, so that no matter how thejoints are bent, the cumulative translation is not enough to shift theassociated sensor region 350 away from the joint region, and also notenough to cause one sensor region to shift into the area of aneighboring joint region. Thus a one-to-one correspondence between jointregions and associated sensor regions is always maintained. This designwill cause only a portion of the FBG to expand or contract with thejoint bending, and thus the associated FBG reflectance spectrum willchange shape with bending, rather than shifting up or down as they wouldif the entire FBG experienced the strain of bending.

Alternately, the known mechanical properties of the fiber 220 and theconstraints on the shape of the fiber 220 when the instrument 200 isbent in various positions can be used to model the shifts in the FiberBragg Grating sensor readings. The joint positions can then be derivedbased on this model. This derivation can be accomplished by analyzingthe shape of the reflection spectrum, which will change if a portion ofthe fiber in the FBG region is bent, but not the entire FBG region.Based on the observable effects on the reflection spectrum caused bybending confined to only a portion of the FBG region, and a forwardmodel of joint angle to reflection spectrum can be derived. This forwardmodel can then be inverted to obtain the joint angle estimate from themeasured reflection spectrum.

In the simplified example shown in FIG. 2, the instrument 200 includesthree rigid body segments 212 coupled by two joint regions 214, with thefirst body segment 212 a coupled to the second body segment 212 b viathe first joint region 214 a, and the second body segment 212 b coupledto the third body segment 212 c via the second joint region 2141). Inone embodiment, the proximal end of the fiber 220 may be attached to thechannel 216 of the instrument 200 at point A1, thereby fixing axialposition of the fiber 220 relative to the instrument 200 at point A1.The distal end of the fiber 220 may be left free to slide axiallyrelative to the channel 216. If the channel 216 is not provided on theneutral axis of the instrument 200, then when the instrument 200 is bentat the first joint region 214 a, the fiber 220 may slide forward orbackward through the channel 216, depending on the direction of rotationrelative to the location of the channel 216. As a result, the sensorregion 350 associated with the second joint region 214 b will translaterelative to the joint region 214 b.

Compensation for the movement of the sensor region 350 may beaccomplished in a variety of ways. For example, the detection of thestrain in the first sensor region 350 associated with the first jointregion 214 a can be used to determine the direction and extent ofrotation of the first joint region 214 a. Based on this information, theexpected location of the second sensor region 350 relative to the secondjoint region 214 b can be determined. This determination can beaccomplished by calculating the axial displacement based onpredetermined models, or by looking up the axial displacement in apre-calibrated table.

This information, in turn, is used in conjunction with the detectedsignals reflected from the second sensor region 350 to determine thedirection and extent of rotation of the second joint region 214 b. Theinformation regarding the axial shifting of the second sensor region 350may be used to compensate for the change in reflected signal caused bythe shifting. Based on the known position and orientation of the firstbody segment 212 a, the calculated direction and extent of rotation ofthe first and second joint regions 214, and the known length of thethird body segment 212 c, the position of the distal end 218 of theinstrument may then be determined. In embodiments having a greaternumber of body segments, a similar determination may be made for eachbody segment based on the information received from the preceding bodysegments.

A similar situation occurs when the bending of a joint region 214 isacute enough that the length of the bent region does not extend over theentirety of the sensor region 350, even though no significant axialtranslation of the fiber 220 relative to the joint region 214 occurs.Because the strained portion of the sensor region 350 does not encompassthe entire length of the sensor region 350, a change in spectral shape,rather than a simple shift of the reflectance spectrum, will occur.

Alternately, the sensor regions 350 may be made short enough that axialtranslation of the fiber 220 will move them partially out of the jointregions 214. As with the case described above, the result of thisbending applied to only part of the sensor regions 350 is that the shapeof the FBG reflectance spectrum will shift, rather than a simple shiftin the center wavelength of the spectrum. In this case, the known axialshift in the fiber 220 at each joint region 214 can be used to assist indetermining the degree of bending of the fiber 220 at each successivejoint region 214. If, for example, the fiber 220 is anchored at theproximal end of the structure, the axial position of the fiber 220 atthe first joint region 214 is fixed, and therefore the effects of thebending of the first joint region 214 on the FBG reflectance spectrum (ashift in the spectrum, a shape change, or a combination of the two), isknown, and a measurement of the reflectance spectrum shape and shift canbe directly used to determine the bend angle of this first joint region214. That bend angle, along with the known structure of the channelcontaining the sensor fiber 220, can be used to determine the degree ofaxial shift in the fiber 220 at the second joint region 214. Therefore,the effects of the bending of the second joint region 214 on the FBGreflectance spectrum of its associated sensor region 350 become known,and the measurement of that reflectance spectrum can be used todetermine the bend angle of the second joint region 214. This procedurecan be iterated for each joint region 214 in the structure serially.

FIG. 10 illustrates a possible spectrum that may result when only partof a sensor region is contained with a joint region. This situation mayarise when the fiber shifts axially, thereby moving part of the sensorregion outside the joint region. Alternatively, this situation may ariseif the sensor region is made long enough so that the entire joint regionis covered by the FBG, but only a portion of the FBG is strained by thejoint motion. As shown in FIG. 10, a sensor region 1050 comprising apair of fiber cores 1030 a-1030 b is positioned so that approximatelyhalf of the sensor region 1050 is inside of the joint region. Therefore,in both fiber cores 1030 a-1030 b, a first portion Ra1, Rb1 that isoutside of the joint region is unstressed, and a second portion Ra2, Rb2which is inside of the joint region is under stress when the jointregion is bent. In this case, Ra2 is compressed and Rb2 is extended.

Each of the plots 1000 a-1000 b show the reflectance spectrum of thesingle Fiber Bragg Grating section in each core 1030 a-1030 b,respectively. The vertical axis corresponds to the intensity of thereflection, and the horizontal access corresponds to wavelength (withwavelength increasing from left to right). The first line 1001 a-1001 bcorresponds to the reflectance spectrum intensity when the fiber isunbent, and therefore each Fiber Bragg Grating is at its nominalwavelength. The second line 1002 a-1002 b corresponds to the reflectancespectrum that would result if the entire length of the Fiber BraggGrating region were bent. The third line 1003 a-1003 b shows thereflectance spectrum corresponding to the partial bending illustrated inFIG. 10.

For fiber core 1030 a, which is partially compressed by the bending, thereflectance spectrum of the bent section shifts to shorter wavelengths,but the partial reflections from this strained section Ra2 combinelinearly with the partial reflections from the unstrained section Ra1.These reflections add coherently to produce the total reflection 1003 a.Because part of the reflected light is shifted and part is not, thereflectance spectrum 1003 a changes in both shape and center pointposition from the default spectrum 1001 a. For the illustrated case,where the shift in the bent section is roughly equal to the spectralwidth of the reflection from the entire grating, the net result is abroadening of the spectrum, as illustrated by the third line 1003 a. Formore extreme shifts, the spectrum may split into two distinct peaks, andbecause the reflections add coherently and the detection system measuresthe intensity of the reflected light, the intensity of each peak will beone quarter of that of the unbent grating (where the grating ispositioned so only half of it is within the joint region). For thesecond fiber core 1030 b, which is partially extended by the bending,the same effect occurs, except that the partial reflection is shifted tolonger wavelengths, as illustrated by the third line 1003 b.

In cases where the fiber core is not uniformly stressed, the peak of thereflection spectrum is not directly proportional to the strain in thefiber core. Similar effects occur for other partially-bent sections ofthe fiber. Therefore, if the fiber cannot be placed on the neutral axisof the structure, or otherwise there is not a one-to-one mapping betweenthe sensor regions and the joint regions, the changes in the shape canbe used along with the shift in the peak wavelength to better determinethe degree of strain in the bent portion of the fiber, and thereby theposition of the joint region.

It is understood that the arrangement may vary in other embodiments. Forexample, the fiber 220 may be fixed to the distal end 218 of theinstrument, rather than at the proximal end, as described above.Alternatively, the fiber 220 may be fixed at any known intermediateposition. In yet other embodiments, the fiber 220 may be fixed atmultiple intermediate and/or end positions. Overall changes in tensioncaused by stretching or contraction of the fiber 220 could be removed ascommon-mode noise, similar to a temperature disturbance. In embodimentsin which the fiber is fixed at two points, it may be desirable to affixthe fiber under tension in the neutral state so that any contractioncaused by bending will result in a decrease of the pre-tension, ratherthan compression of the fiber. In addition, the number, shape, and sizeof the body segments 212 may vary. For example, fewer or greater jointregions 214 may be provided, thereby increasing the number of bodysegments 212. In addition, the body segments 212 need not be linear inshape and may include rigid curves or angles.

In accordance with other embodiments of the present invention, one ormore of the body segments may comprise a flexible material. In thiscase, the bending of the instrument 200 is not limited to solely thejoint regions 214. Thus, the constraints imposed on the fiber by theflexible structure may be used to model the system. This model may thenbe inverted either analytically or numerically to derive theinstrument's state from the detected set of signals reflected from theFiber Bragg Gratings. In other words, a forward model is generated thatcomputes the strains expected at the FBG regions from an arbitrary butphysically realizable bending of the structure. For example, thestiffness of the flexible section might force it to take the shape of asmooth spline curve when it experiences external forces at the ends andat points along its length, but not allow it to take a right-angle bendin the middle. In the forward model, the input would be the structure'sdegree of bending at each point, and the output is the predicted strainsat the FBG regions. Inverting this model provides another model, thisone taking as input the strains at the FBG regions, and providing asoutput the structure's degree of bending. In some cases this may be doneanalytically, but for complicated systems, the computation would be donenumerically, in real time.

FIG. 5 is a simplified block diagram of another embodiment of thepresent invention in which a surgical instrument 500 is provided havingan optical fiber 520, similar to optical fiber 220 described above. Inthis embodiment, the optical fiber 220 is axially fixed at two anchorpoints 502-503, with at least one Fiber Bragg Grating sensor region 540provided between the two fixed anchor points 502-503 to form a fixedaxial position region 560. Strain measurements may then be made at thissensor region 540 for a variety of purposes.

For example, the fixed position region 560 of the instrument 500 may bemade of a compressible material, such as rubber, or include acompressible portion, such as a torsion spring. Thus, when the distaltip of the instrument 500 is used to apply a force in the axialdirection onto a surface, the compressible fixed position region 560will compress, thereby compressing the fiber 520. The sensor region 540may be used to detect the axial loading of the fiber 520 in the fixedposition region 560, thereby enabling determination of the force appliedby the tip of the instrument 500. This determination of force may alsoinclude a determination of torque as well.

In other embodiments, the fixed position region 560 is substantiallyrigid so that no external load is applied to the portion of the fiber520 in the fixed position region 560. In this case, changes intemperature of the fiber 520 and the rigid section 560 cause thermalexpansion or contraction of the fiber 520 and the rigid section 560.Because the fiber 520 is fixed at anchor points 502-503, the net thermalexpansion of this composite structure causes strain in the fiber 520.This strain combined with the temperature-induced change in therefractive index causes a shift in the Fiber Bragg Grating's reflectionpeak, which can then be detected as described above. The temperature maybe determined by looking up a detected signal in a pre-calculated lookuptable, thereby eliminating the need for comparison with the secondsensor region.

In some embodiments, a second sensor region is provided adjacent to thefiber 520 in a second fiber that is not bounded by fixed anchor points.Thus, thermal expansion of the second fiber in this location causesaxial movement of the second fiber and does not cause strain in thesecond fiber. The two fibers are placed close together, so they are atsubstantially the same temperature. The common-mode signal in each fiberis a function of both the temperature and the strain in each fiber, andsince the second fiber is free to move axially, it experiences no strainand thereby provides an output that is a function of temperature only.The shift in the reflected signal from the second fiber can then besubtracted from the shift in the reflected signal from the first fiber,allowing determination of both the temperature and axial strain inregion 540 independently.

In yet other embodiments, multiple sensor regions may be provided alongthe length of the fiber 520, thereby enabling the instrument 500 to beused for multiplexed temperature sensing. In yet other embodiments, thetemperature may be measured in the sections of the fiber correspondingto joint regions. The shift in wavelength depends on both the strain inthe fiber core, and on the temperature of the fiber core. In thebend-sensitive sections of the fiber, there is no axial strain (whichcauses a similar wavelength change in all the FBGs in a sensing region,and hence a common-mode signal). In these sections, the bending straincauses opposite shifts in the wavelength of the FBGs on opposite sidesof the core, and hence a differential-mode signal. Adding the wavelengthshifts for all the FBGs in a single sensitive region thus nulls thebending signal, and amplifies the temperature signal. Thus, thetemperature measurement may be accomplished by using the sum of theoppositely disposed cores' readings rather than the difference, as longas there is no axial loading of the fiber.

In these temperature sensing embodiments, it may be desirable for thebody segments in the fixed position region 560 to be made of a thermallyconductive material, such as a metal, or for at least a portion of thefiber 520 to be exposed to the exterior of the instrument 560 so as tobe placed in contact with the target environment for which temperaturedetection is desired.

In the embodiment shown in FIG. 5, the fixed position region 560 isprovided at the distal end of the instrument 500. In other embodiments,one or more fixed position regions 560 including Fiber Bragg Gratingsensor regions may be provided at different locations along the lengthof the instrument 500. These additional sensor regions may be used,e.g., to measure strain at joint regions, as described above withrespect to FIGS. 2-4. Thus, the same fiber 520 may be used to bothdetect bending at the joint regions and force or temperature at thefixed position regions.

In accordance with other embodiments of the present invention, the fibercontaining Fiber Bragg Gratings may also be used for other purposes,such as to provide illumination in a body cavity. For example, if thedistal end of the fiber is exposed at the distal end of a surgicalinstrument, such as an endoscope, the fiber may be used to provideillumination for the endoscope. The Fiber Bragg Grating sensorstypically operate in a small band in the infrared region of the lightspectrum, e.g., 1.55 μm wavelength. In this case, the fiber may also beused to convey light for illumination to the end of the endoscopewithout interfering with the operation of the strain sensors, as long asthe illuminating light is sufficiently removed in wavelength that theilluminating light can be filtered from the infrared light used tointerrogate the Fiber Bragg Grating sensors. The illuminating light maybe provided in the visible range to directly illuminate a scene.

FIG. 6 is a perspective view of a surgical instrument 600 in accordancewith embodiments of the present invention. This surgical instrument 600may be used as part of a robotic surgical system, such as the roboticsurgical system 2 described above or with the da Vinci Surgical Systemby Intuitive Surgical, Inc. of Sunnyvale, Calif. A similar roboticmanipulator and surgical instrument is described in U.S. Pat. No.6,902,560, the disclosure of which is incorporated by reference hereinin its entirety.

The surgical instrument 600 includes a shaft portion 602, a controlhousing 603 provided at a proximal end of the shaft portion 602, and aworking end 610 at a distal end 604 of the shaft portion 602. Theworking end 610 comprises three joint assemblies 620 a-620 c couplingthree body segments 612 a-612 c to the distal end 604 of the shaftportion 602. An end effector 630 is coupled to the third body segment612 c via a wrist assembly 622 at the distal end of the instrument 600.The end effector 630 is the working part of the surgical instrument 600and can include clamps, jaws, scissors, gaspers, needle holders,micro-dissectors, staple appliers, tackers, suction irrigation tools,and clip appliers, and non-articulated tools, such as cutting blades,cautery probes, irrigators, catheters, and suction orifices.

In this embodiment, each of the joint assemblies 620 a-620 c provideone, two, three, or more degees of freedom. An optical fiber, such asthe fiber 220 described above, may be provided in a channel through theshaft portion 602, the joint assemblies 620 a-620 c, and the bodysegments 612 a-612 c, in order to provide position information regardingthe working end 610 and, more particularly, the end effector 630 of theinstrument 600. Because the shaft portion 602 is substantially rigid,the position and orientation of the distal end 604 can be determinedbased on the known position and orientation of the proximal end of theshaft, which is under the control of the manipulator assembly of therobotic system. Therefore, the optical fiber need not be used todetermine the position of the shaft portion 602 and Fiber Bragg Gratingsneed not be provided along the portion of the fiber within the shaftportion 602.

Although it may be possible to estimate the position of the end effector622 based on the control inputs to the joint assemblies 620 a-620 c,errors in the estimated position may be introduced due to stretching ofthe joint actuation cables or other conditions of the driving mechanism.Therefore, sensor regions of the fiber may be provided at each jointassembly 620 in order to determine the extent and orientation of anybending at that joint. This can, in turn, be used to determine theposition and orientation of subsequent body segments 612 and ultimatelythe end effector 630. Roll movement of body segments may be accommodatedby providing the sensing fiber in a channel so as to allow rotation ofthe fiber within the channel. The fiber is quite rigid in torsion, androtates freely inside the channel, so that roll movement of the bodysegments would not cause rolling or twisting of the fiber. The extent ofrolling may be measured using conventional actuator sensors. Thisrotational information may then be combined with the informationobtained from the bend sensors to determine the orientation of thedistal links in space, and to transform the two-dimensional bendingdetermined by the fiber sensor into the degrees of freedom that areactuated at each joint.

In accordance with one aspect of the present invention, the informationregarding joint bending obtained from the FBG bend sensors may be usedas part of the feedback loop for the control system 250. As describedabove, actuator sensors may be used to measure the rotation of themotors used to actuate linear movement of the cables, which, in turn,actuates rotational movement of the actively controlled joints. Due toerrors introduced by any structures or drivetrain mechanisms that aredistal to the actuator sensors, the actual rotation of the joints maynot correspond to the rotation measured by the actuator sensors. Thebending detected by the bend sensors located in those joints can then beprovided back to the servo controller 16 as part of a feedback loop toimprove the control of the motors.

In accordance with another aspect of the present invention, FBG sensorsmay be used in a surgical instrument including both passive andactively-controlled joint regions. FIG. 7 is a simplified perspectiveview of a surgical instrument 700 in accordance with embodiments of thepresent invention. The instrument 700 includes a passive region P and anactive region A. The passive region P may comprise, e.g., an elongateshaft made of a flexible material and/or an elongate shaft comprising aseries of rigid segments 702 coupled by joint regions 704. Similarshafts of rigid segments 702 coupled by joint regions 704 have been usedin various endoscopes currently on the market. The shaft in the passiveregion may be, e.g., approximately 2 m long and 12 mm in diameter. Inother embodiments, the dimensions may vary.

Unlike the shaft portion 602 in FIG. 6, the passive region P is notrigid and may bend to facilitate insertion through body passageways andaround structures in the patient's body. However, the bending at eachjoint region 704 is not directly controlled by the operator.

At the distal end 706 of the passive region P is a working end 710,which forms the active region A of the instrument 700. In thisembodiment, the working end 710 comprises a pair of actively-controlledrobotic arms 740 and an imaging device (e.g., a camera 750). In otherembodiments, the working end 710 may have greater or fewer components.For example, the working end 710 may include only a single arm 740 orthree or more arms 740, and the imaging device may be omitted.

Each of the robotic arms 740 may be similar in operation to the workingend 610 described above with respect to FIG. 6. Each arm 740 comprisesthree joint assemblies 720 a-720 c coupling three body segments 712a-712 c to the distal end 706 of the passive region P. An end effector730 is coupled to the third body segment 712 c via a wrist assembly 722at the distal end of the instrument 700. As with instrument 600, each ofthe joint assemblies 720 a-720 c may provide one, two, three, or moredegrees of freedom.

The instrument 700 may be inserted into a patient's body and controlledusing a robotic surgical system similar to the robotic surgical system 2shown in FIG. 1. In this case, only a single manipulator assembly may beused to control the single instrument 700. In other cases, multiplemanipulator assemblies may be used to control other instruments.

As with the surgical system 2 described above, the movement of eachjoint assembly 720 a-720 c may be actuated by motors controlled by aservo controller (not shown). The motors may be coupled to the jointassemblies 720 a-720 c using, e.g., actuation cables. In instrument 600,the shaft 602 is rigid and linear, so the joint assemblies 620 a-620 cmay be actuated using a combination of rigid shafts and cables made of astiff material, such as, e.g., tungsten. In contrast, in instrument 700,the working end 710 is separated from the proximal end 705 of theinstrument 700 by a flexible region. As a result, rigid shafts cannot beused to transmit the actuation force, and stiff cables may unacceptablylimit the bending of the passive region P and experience high frictionalforces when being linearly translated through a curved path. Thus, theactuation cables may comprise a less rigid material, such as a polymeror composite.

Although a polymer cable may not experience as much friction as atungsten cable when passed through a channel in the passive region P,the polymer cable may experience significantly increased stretching,thereby increasing the difficulty of measuring the position of eachjoint assembly 720 a-720 c based on measurements of the actuatorsensors. In addition, because the passive region P is not directlycontrolled by the operator, it is difficult for the operator toaccurately determine the position and orientation of the distal end 706inside of the patient's body. Thus, even if the positions of each jointassembly 720 a-720 c were capable of being accurately measured, theposition and orientation of the end effectors 730 would only be knownrelative to the position and orientation of the distal end 706. Thus,the position and orientation of the end effectors 730 relative to anexternal coordinate system (e.g., defined by base 701 in a knownlocation outside of the patient's body) will also be indeterminable.

In accordance with embodiments of the present invention, a multicoreoptical fiber, such as the fiber 220 described above, may be provided ina channel through the passive region P and/or the active region A, inorder to provide position information of the instrument 700, asdescribed above. In one embodiment, the fiber may be provided only inpassive region P, so that the position and orientation of the distal end706 can be determined relative to the external coordinate system. Theposition and orientation of the end effectors 730 relative to the distalend 706 can then be determined based on the measurements by the actuatorsensors.

In another embodiment, the fiber including FBG bend sensors may beprovided only in the active region A. The position and orientation ofthe distal end 706 of the passive region P may be determined using anyof the conventional methods described above. In this case, the positionand orientation of each individual segment along the length of thepassive region P may be irrelevant, so long as the position andorientation of the distal end 706 is known. Thus, it may be possible touse an external detection system to determine the position andorientation of the distal end 706. The FBG sensors in the fiber can thenbe used to determine the position and orientation of each joint assembly720 a-720 c in the arms 740. This information can be combined with theexternally detected position of the distal end 706 to provide accurateposition and orientation information regarding the end effectors 730.

In yet another embodiment, the fiber including FBG bend sensors may beprovided along the entire length of the instrument 700. Thus, theposition and orientation along the entire length of the instrument 700can be determined relative to an external coordinate system.

This position and orientation information can be used for a variety ofpurposes. For instance, the operator may utilize the determined positionof the end effectors 730 in order to navigate to a site of interest inthe patient's body. Alternatively, the control system may utilize thisinformation as part of a feedback loop.

FIG. 8 is a simplified block diagram illustrating a dual loop control,multi-input/single output feedback loop 800 in accordance withembodiments of the present invention. The dual-loop control utilizes twofeedback sources, the actuator sensor feedback 807 and the feedback 808from the bend sensors in the joint assemblies, in order to compensatefor accuracy problems that arise from errors occurring between motor 804and load 806. The actuator sensor feedback 807 is used to close thevelocity loop 803 and the bend sensor feedback 808 is used to close theposition loop 802.

Because the load 806 is connected to the motor 804 through thedrivetrain 805 which is necessarily compliant, the load 806 does notreact immediately to changes in motor position. The time lag betweenmotor rotation and load movement can make the load feedback too slow toclose the velocity loop 803. In a simple case, one solution is toconnect the bend sensor, which directly senses the load, to the slowerposition loop 802 (where accuracy is most important), and to connect theactuator sensor to the faster velocity loop 803 (where responsiveness ismost important). In other cases, the dual loop feedback may utilize morecomplex algorithms for mixing the two sensor signals. In addition,unmodeled friction in the drivetrain can make the position and velocityof the load a poor indicator of the position and velocity of the motor,and thus inappropriate for use as an actuator sensor.

In accordance with other aspects of the present invention, FBG sensorsmay be used to locate a portion of a surgical instrument (e.g., the tipof the instrument) in a fixed external coordinate system in order tocombine external data (such as image data from a magnetic resonanceimaging (MRI) or ultrasound imaging system) with the positioninginformation to assist the operator in performing the surgical procedure.

For example, the detection system 260 may be coupled to the imagingsystem 270 to provide real-time position information for use in thecontrol of the instrument 200. The imaging system 270 may include imagedata of the patient from an external source, such as a pre-operativeMRI. The proximal end of the instrument 200 can then be supported by thesurgical system 2 in a known position relative to the patient. Thus, oneend of the sensing fiber may have a fixed position and angularorientation in a coordinate system external to the patient. For example,in a surgical robot, the proximal end of the fiber may be fixed to theframe holding the apparatus, which would be fixed in position relativeto the patient during a period of use. As described above, the detectionsystem 260 can then utilize the FBG sensors to determine the angles ofall the joint regions in the instrument, including those that are notactively controlled. This information can then be combined with theknown lengths of the structure's links to determine the position andorientation of each portion of the instrument.

A calibration step may be used, depending on whether the external datato be merged is available in the coordinate system of the frame thatholds the fixed end of the fiber. In some cases (such as forpre-operative MRI images), the image data will be referenced tolandmarks of patient anatomy. The image data can be aligned byidentifying three patient landmarks that are visible in the externaldata with the robotic camera, and viewing these from two differentangles. The landmarks may be internal or external anatomical features,or fiducials placed prior to imaging.

The sensor system, by providing in three dimensions unambiguous positionand orientation information for each link in the instrument, enablesthis coordinate mapping for all parts of the instrument. This takesadvantage of the ability of the OFDR interrogation system to multiplexmany FBG sensors along a single fiber. In addition to the imageregistration described above, this allows the system to provide a richerset of features to the user. For example, the position of tools or otherstructure elements that are not currently in the field of view of acamera can be determined, and indicated to the user, and these featurescan be provided more easily across a wide range of minimally invasivesurgery platforms with different architectures.

In some embodiments, the position information can be combined with theexternal image data to provide the operator with an image showing thelocation of the tip of the instrument on the MRI image in order toprovide additional context for the surgical procedure.

In accordance with other embodiments, the external data may be combinedwith the image produced by a camera carried on the surgical instrument,such as, e.g., camera 750 in FIG. 7. If the surgical system is one inwhich the camera is part of the same structure as the manipulators (asin the flexible “snake-like” instrument shown in FIG. 7), then the sameposition data used to provide enhanced feedback control of the roboticarms 740 can also be used to determine the position of the camera 750.This data can be combined with other data on the orientation of theproximal end 705 of the fiber (anchored to the instrument's mountingstructure) relative to the patient, to determine the location andorientation of the endoscopic camera 750 with respect to the patient.Once this is known, the MRI or other data, which is available in acoordinate system referenced to the patient, can be superimposed on thecamera image, or displayed alongside it, viewed from the same point asthe camera image. If the camera 750 is controlled on a separate roboticarm, a fiber sensor of the type described above can be added to therobotic arm to enable these operations.

In accordance with other embodiments of the present invention, FBG bendsensors in a surgical instrument may be used for calibration purposes inorder to correct for manufacturing defects. These manufacturing defectsmay affect the dimensions and control of the various components of asurgical instrument, thereby causing the end effector and anyintermediate components to be located in a position other than theassumed ideal. In order to calibrate the device, the control system willattempt to position the instrument in one or more known configurations.The FBG bend sensors in the instrument may be used to compare themeasured position of the instrument to the expected position of theinstrument. Any differences between the measured position and theexpected position can be then be compensated for by the controllersoftware.

The sensing of the position and orientation of the distal end of asurgical instrument enables the performance of a system identificationon each device's actual kinematics, mapping out errors (e.g.,differences from the ideal kinematics, such as those induced bymanufacturing or material variation) automatically, and storing thiserror map in non-volatile memory on the device. Thus, the accuracy ofthe device can be improved. This mapping might occur once in thefactory, or might occur each time the device is used. The latter wouldallow for correction of errors induced by aging or autoclave cycles, orany other time-dependent cause.

FIGS. 9A-9B are simplified block diagrams illustrating top and sideviews of a portion of a surgical instrument 900, in accordance withother embodiments of the present invention. The surgical instrument 900comprises a plurality of rigid links 912 coupled by a plurality of jointregions 920 having an end effector 930 provided at the distal end. Insome cases (as in the embodiment shown in FIGS. 9A-9B), the proximal endof the instrument 900 may be mounted to a fixed base. In other cases,the proximal end of the instrument 900 may be mounted to a distal end ofan arm, such as the shaft portion 602 shown in FIG. 6. In thisembodiment, each joint region enables passive bending in one plane. Forexample, the joint regions 920′ (marked “A”) allow rotation about anaxis in the z-direction, and the joint regions 920″ (marked “B”) allowrotation about an axis in the y-direction. Each of the joint regions920′ and 920″ allows a range of motion of, e.g., ±45°. A plurality ofoptical fiber cores including at least one Fiber Bragg Grating regionare provided in the instrument 900. The fiber cores may be part of asingle optical fiber anchored to the base and passed through the neutralaxis of the instrument 900. The FBG regions in the fiber cores may beused to determine the position and orientation of the instrument 900,similar to the embodiments described above.

Embodiments of the present invention may provide various advantages notprovided by prior art systems. For example, embodiments may enable moreprecise control of the elements located at the distal end of arobotically-controlled surgical instrument. In other embodiments,precise positioning and control may be achieved of arobotically-controlled surgical instrument having unactuated or flexiblebends or joints.

In some embodiments, the cross-sectional area of the surgical instrumentmay be reduced because a single fiber can be used for multiple purposes,including sensing the position of all of the joints in the structure,sensing the forces applied to the structure, sensing temperature, andproviding illumination for an imaging system. This may have particularapplicability for endoscopes having a limited number of working ports.

In some embodiments, the angles of each of the joints or flexibleportions of the surgical instrument may be mapped in real time. This canenable precise transformation of the location coordinate system at theinstruments end effector to a global coordinate system. Thistransformation may enable the combination of data and images acquired byother systems with data acquired by the surgical system, such as, e.g.,images obtained from endoscopic cameras mounted on the end effector.

In addition, by determining the position and orientation of the segmentsof the surgical instrument in a world reference frame, a flexiblerobotic surgery system can more completely mimic the operation ofpre-existing minimally-invasive surgical systems that are designed toprovide this information, such as, the da Vinci Surgical System,providing the ability to re-use user features and the code basedeveloped for those systems. Thus a consistent surgeon experience can beprovided, independent of the robot system architecture.

While the invention has been described in terms of particularembodiments and illustrative figures, those of ordinary skill in the artwill recognize that the invention is not limited to the embodiments orfigures described. For example, the position

in many of the embodiments described above, the optical fiber comprisesa single fiber having multiple cores. In other embodiments, the multiplesingle-core optical fibers may be used.

Embodiments described above include the use of actuator sensors fordetecting movement of the actuators. In various embodiments, theactuator sensors may detect one or more of the position, velocity,acceleration, and applied force of the actuators. It will be understoodthat the detected applied force may be an applied torque, and thedetected velocity and acceleration may be an angular velocity andangular acceleration. In addition, the sensed information provided bythe actuator sensors to the control system may be provided in variousforms and need not be limited to a direct measurement of position,velocity, acceleration, and applied force. For example, the actuationsensor may provide a logarithm of position to the control system, which,in turn, converts that value into a position.

In addition, the robotic surgical system described above may utilizecables in tension in order to actuate movement of the surgicalinstrument. In other embodiments, other actuation mechanisms, such asrods or tubes in compression and torsion, may be used to actuatemovement. In addition, the FBG sensors may be used to detect the bendingof passive joints as well. Some embodiments may have particularapplicability for systems utilizing hydraulic or pneumatic mechanisms toactuate movement. Although hydraulic and pneumatic mechanisms are notsubject to the frictional forces afflicting cables or tubes, thesemechanisms introduce other control problems. For instance, in order toutilize hydraulics or pneumatics to transmit actuation forces through aflexible or multi-jointed structure having multiple degrees of freedom,very small cylinders that may not be completely sealed are used. The useof such small cylinders may result in a drift in the expected positionof the actuator. Having real-time position feedback of the positions ofthe hydraulically or pneumatically actuated joints would ameliorate thisproblem.

In addition, the anchoring of the optical fiber cores relative to theinstrument body may be accomplished in a variety of ways. For example,the optical fiber may be attached to an interior surface of the channelpassing through the instrument. Alternatively, the optical fiber may beattached to a structure external to the robotic arm, but at a fixedposition relative to the arm.

Therefore, it should be understood that the invention can be practicedwith modification and alteration within the spirit and scope of theappended claims. The description is not intended to be exhaustive or tolimit the invention to the precise form disclosed. It should beunderstood that the invention can be practiced with modification andalteration and that the invention be limited only by the claims and theequivalents thereof.

What is claimed is:
 1. A method comprising: determining, in a coordinatesystem referenced to a patient, a location of a camera carried on aminimally invasive surgical instrument based on information from anoptical fiber sensor in an optical fiber in the minimally invasivesurgical instrument, wherein the camera has a first point of view basedon a position and an orientation of the camera; and displaying, relativeto the coordinate system referenced to the patient, a first image of thepatient from the camera and a second image of the patient from a sourcedifferent from the camera, wherein the first image is from the firstpoint of view and the second image is from the first point of view andwherein the first image and second image are displayed concurrently on acommon display device.
 2. The method of claim 1 wherein the determiningfurther comprises: determining the position of the camera based on theinformation from the optical fiber sensor; and generating the positionand orientation of the camera in the coordinate system referenced to thepatient by combining the determined position of the camera withorientation data of a proximal end of the optical fiber wherein theproximal end of the optical fiber has a fixed position and orientationin the coordinate system referenced to the patient.
 3. The method ofclaim 1, wherein the image from the camera shows a location of a tip ofthe minimally invasive surgical instrument.
 4. The method of claim 1,wherein the optical fiber sensor comprises a Fiber Bragg Grating region.5. The method of claim 1, wherein the optical fiber includes an opticalfiber sensor in one or more joint regions of the surgical instrument. 6.The method of claim 5, wherein joint regions of the surgical instrumentinclude an actively controlled joint region and a passively bendablejoint region.
 7. The method of claim 5, wherein the determining aposition of the camera further comprises: determining an angle of the oreach joint region of the surgical instrument.
 8. The method of claim 7,wherein the determining a position of the camera further comprises:combining the or each determined angle with known lengths of links ofthe surgical instrument to determine a position and orientation of eachportion of the surgical instrument.
 9. The method of claim 1, furthercomprising: determining, in the coordinate system referenced to apatient, a position of a distal tip of a second surgical instrumentbased on information from an optical fiber sensor in an optical fiber inthe second surgical instrument, wherein the distal tip of the secondsurgical instrument is not in the field of view of the camera.
 10. Themethod of claim 9, wherein the displaying further comprises: displayingthe position of the distal tip of the second surgical instrument. 11.The method of claim 9: wherein the second surgical instrument comprisesone or more joint regions; and wherein the optical fiber of the secondsurgical instrument includes an optical fiber sensor in the or eachjoint region of the second surgical instrument.
 12. The method of claim11, wherein the second surgical instrument comprises an activelycontrolled joint region and a passively bendable joint region.
 13. Themethod of claim 11, wherein the determining the position of the distaltip of a second surgical instrument further comprises: determining anangle of the or each joint region of the second surgical instrument. 14.The method of claim 13, wherein the determining the position of thedistal tip of a second surgical instrument further comprises: combiningthe or each determined angle of each of the joint regions of the secondsurgical instrument with known lengths of links of the second surgicalinstrument to determine a position and orientation of each portion ofthe second surgical instrument.
 15. The method of claim 1 wherein thefirst image is displayed adjacent to the second image.
 16. The method ofclaim 1 wherein one of the first and second images is superimposed onthe other of the first and second images.